Systems and methods for autofocus and depth map generation

ABSTRACT

Aspects of the present disclosure relate to systems and methods for performing an autofocus operation. An example device may include one or more processors and a memory. The memory may include instructions that, when executed by the one or more processors, cause the device to determine a focus value for each of a plurality of focal regions for a camera set to a focal length, and determine, based on the focus values, one or more focal regions of the plurality of focal regions to exclude from use in triggering or performing an autofocus operation.

TECHNICAL FIELD

This disclosure relates generally to systems and methods for imagecapture devices, and specifically to generating depth maps and autofocusoperations for a camera.

BACKGROUND OF RELATED ART

Devices including or coupled to one or more digital cameras use a cameralens to focus incoming light onto a camera sensor for capturing digitalimages. The camera lens has a focal length that places a range of depthof the scene in focus. Portions of the scene closer or further than therange of depth may be out of focus, and therefore appear blurry in aresulting image. The distance of the camera lens from the camera sensorindicates the distance of the range of depth for the scene from thecamera lens that is in focus. Many devices are capable of moving thecamera lens to adjust the distance between the camera lens and thecamera sensor, thereby adjusting which portions of a scene appear infocus for captured images.

A device may attempt to determine the position of the camera lens toplace a portion of interest of the scene in focus. In one example, auser may touch an area of a preview image provided by the device (suchas a person or landmark in the previewed scene) to indicate the portionof the scene to be in focus. In another example, the device mayautomatically determine a portion of the scene to be in focus. Inresponse, the device may perform autofocus (AF) operations toautomatically adjust the camera lens position so that the portion of thescene is in focus for subsequent image captures.

Changes in the scene, such as the user moving the camera or objectsentering or exiting the scene, may cause the device to determine thatthe camera lens position may need to be adjusted. For example, an objectentering the scene may cause a device to retrigger AF operations fordetermining a camera lens position. For some AF operations, devices mayuse information sensed by the camera sensor or a separate ranging systemin creating a depth map for the scene. The depth map indicates depths ofdifferent portions of a scene from the camera or ranging system. Thedepth map may be used for AF operations as well as other operations,such as object recognition and detection.

SUMMARY

This Summary is provided to introduce in a simplified form a selectionof concepts that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tolimit the scope of the claimed subject matter.

Aspects of the present disclosure relate to systems and methods forperforming an autofocus operation. An example device may include one ormore processors and a memory. The memory may include instructions that,when executed by the one or more processors, cause the device todetermine a focus value for each of a plurality of focal regions for acamera set to a focal length, and determine, based on the focus values,one or more focal regions of the plurality of focal regions to excludefrom use in triggering or performing an autofocus operation.

In another example, a method is disclosed. The example method includesdetermining a focus value for each of a plurality of focal regions for acamera set to a focal length, and determining, based on the focusvalues, one or more focal regions of the plurality of focal regions toexclude from use in triggering or performing an autofocus operation.

In a further example, a non-transitory computer-readable medium isdisclosed. The non-transitory computer-readable medium may storeinstructions that, when executed by a processor, cause a device toperform operations including determining a focus value for each of aplurality of focal regions for a camera set to a focal length, anddetermining, based on the focus values, one or more focal regions of theplurality of focal regions to exclude from use in triggering orperforming an autofocus operation.

In another example, a device is disclosed. The device includes means fordetermining a focus value for each of a plurality of focal regions for acamera set to a focal length, and means for determining, based on thefocus values, one or more focal regions of the plurality of focalregions to exclude from use in triggering or performing an autofocusoperation.

Other aspects of the present disclosure relate to systems and methodsfor generating a depth map. An example device may include one or moreprocessors and a memory. The memory may include instructions that, whenexecuted by the one or more processors, cause the device to receive afirst intensity measurement for each of a plurality of focal regions fora camera at a first focal length, determine, from the first intensitymeasurements, a first focus value for each of the plurality of focalregions, generate a depth map of an indication of focal lengths for theplurality of focal regions based on the first focus values for theplurality of focal regions, and use the depth map in determining when toperform or in performing an autofocus operation for the camera.

The depth map of the indication of focal lengths may indicate for eachfocal region a focal length of the camera at which a portion of a scenecorresponding to the focal region is in focus for the camera.Additionally, the first focus value may be a contrast or a phasedifference.

The instructions, when executed by the one or more processors, mayfurther cause the device to adjust the first focal length of the camera,receive a second intensity measurement for each of the plurality offocal regions for the camera at the adjusted focal length, determine,from the second intensity measurements, a second focus value for each ofthe plurality of focal regions, compare, for each of the plurality offocal regions, the first focus value and the second focus value, anddetermine, based on the comparison, the focal length of the camera atwhich the portion of the scene corresponding to the focal region is infocus for the camera. The first intensity and the second intensity maybe light intensities.

The instructions, when executed by the one or more processors, mayfurther cause the device to generate, for each focal region, an array offocus values at different focal lengths and include the first focusvalue at the first focal length and the second focus value at theadjusted focal length. Comparing the first focus value to the secondfocus value may include determining one focus value from the array offocus values that is a highest contrast from the array of focus valueswhen the focus values are contrasts or a phase difference (PD) closestto zero from the array of focus values when the focus values are PDs.Additionally, the focal length of the camera at which the portion of thescene corresponding to the focal region is in focus for the cameracorresponds to a position of the determined one focus value from thearray of focus values. Each array of focus values may include a numberof positions equal to a number of different focal lengths for thecamera.

The instructions, when executed by the one or more processors, mayfurther cause the device to generate, based on the first focus values, afirst depth map of focus values for the first focal length, andgenerate, based on the second focus values, a second depth map of focusvalues for the adjusted focal length. Generating the array of focusvalues for a focal region may include populating the array with focusvalues from a same position in the first depth map of focus values andthe second depth map of focus values.

In another example, a method is disclosed. The example method includesreceiving a first intensity measurement for each of a plurality of focalregions for a camera at a first focal length, determining, from thefirst intensity measurements, a first focus value for each of theplurality of focal regions, generating a depth map of an indication offocal lengths for the plurality of focal regions based on the firstfocus values for the plurality of focal regions, and using the depth mapin determining when to perform or in performing an autofocus operationfor the camera.

In a further example, a non-transitory computer-readable medium isdisclosed. The non-transitory computer-readable medium may storeinstructions that, when executed by a processor, cause a device toperform operations including receiving a first intensity measurement foreach of a plurality of focal regions for a camera at a first focallength, determining, from the first intensity measurements, a firstfocus value for each of the plurality of focal regions, generating adepth map of an indication of focal lengths for the plurality of focalregions based on the first focus values for the plurality of focalregions, and using the depth map in determining when to perform or inperforming an autofocus operation for the camera.

In another example, a device is disclosed. The device includes means forreceiving a first intensity measurement for each of a plurality of focalregions for a camera at a first focal length, means for determining,from the first intensity measurements, a first focus value for each ofthe plurality of focal regions, means for generating a depth map of anindication of focal lengths for the plurality of focal regions based onthe first focus values for the plurality of focal regions, and means forusing the depth map in determining when to perform or in performing anautofocus operation for the camera.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the present disclosure are illustrated by way of example, andnot by way of limitation, in the figures of the accompanying drawingsand in which like reference numerals refer to similar elements.

FIG. 1 is a depiction of an example scene to be captured by a camera.

FIG. 2A is a depiction of an example camera lens at a focal length sothat an object is in focus for phase difference AF.

FIG. 2B is a depiction of an example camera lens at too long of a focallength so that the object is out of focus for phase difference AF.

FIG. 2C is a depiction of an example camera lens at too short of a focallength so that the object is out of focus for phase difference AF.

FIG. 3 is a depiction of an example correlation between focal length andphase difference for phase difference AF.

FIG. 4A is a depiction of an example camera lens at a focal length sothat an object is in focus for contrast detection AF.

FIG. 4B is a depiction of an example camera lens at too long of a focallength so that the object is out of focus for contrast detection AF.

FIG. 4C is a depiction of an example camera lens at too short of a focallength so that the object is out of focus for contrast detection AF.

FIG. 5 is a depiction of an example correlation between focal length andcontrast for contrast detection AF.

FIG. 6 is a depiction of an example image of the scene in FIG. 1.

FIG. 7 is a depiction of an example image of the scene in FIG. 1 with aportion of the scene blocked.

FIG. 8 is a depiction of the example image in FIG. 7 and illustratingexample portions for determining a focus value corresponding to anexample region of interest in the scene.

FIG. 9 is a block diagram of an example device for determining when toperform an AF operation, performing an AF operation, and/or generating adepth map.

FIG. 10 is an illustrative flow chart depicting an example operation forexcluding one or more focal regions for triggering and/or performing anAF operation for a camera.

FIG. 11 is an illustrative flow chart depicting an example operation forgenerating a depth map.

FIG. 12 is a depiction of example focus values determined for the focalregions.

FIG. 13 is a depiction of example focus values for the focal regions atan adjusted focal length.

FIG. 14 is a depiction of example focus values for the focal regions atanother adjusted focal length.

FIG. 15 is a depiction of an example array generated for a first focalregion.

FIG. 16 is a depiction of an example depth map of focal lengths for thefocal regions for FIG. 12 through FIG. 14.

FIG. 17 is an illustrative flow chart depicting an example operation forusing a generated depth map of focal lengths to determine which focalregions to exclude.

FIG. 18 is an illustrative flow chart depicting an example operation fordetermining which focal regions to exclude.

DETAILED DESCRIPTION

Aspects of the present disclosure may be used for determining whether totrigger an AF operation or for performing an AF operation. Aspects ofthe present disclosure also may be used for generating depth maps for ascene. In the following description, numerous specific details are setforth, such as examples of specific components, circuits, and processesto provide a thorough understanding of the present disclosure. The term“coupled” as used herein means connected directly to or connectedthrough one or more intervening components or circuits. Also, in thefollowing description and for purposes of explanation, specificnomenclature is set forth to provide a thorough understanding of thepresent disclosure. However, it will be apparent to one skilled in theart that these specific details may not be required to practice theteachings disclosed herein. In other instances, well-known circuits anddevices are shown in block diagram form to avoid obscuring teachings ofthe present disclosure. Some portions of the detailed descriptions whichfollow are presented in terms of procedures, logic blocks, processingand other symbolic representations of operations on data bits within acomputer memory. In the present disclosure, a procedure, logic block,process, or the like, is conceived to be a self-consistent sequence ofsteps or instructions leading to a desired result. The steps are thoserequiring physical manipulations of physical quantities. Usually,although not necessarily, these quantities take the form of electricalor magnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated in a computer system.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the present application,discussions utilizing the terms such as “accessing,” “receiving,”“sending,” “using,” “selecting,” “determining,” “normalizing,”“multiplying,” “averaging,” “monitoring,” “comparing,” “applying,”“updating,” “measuring,” “deriving,” “settling” or the like, refer tothe actions and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (electronic) quantities within the computer system's registersand memories into other data similarly represented as physicalquantities within the computer system memories or registers or othersuch information storage, transmission or display devices.

In the figures, a single block may be described as performing a functionor functions; however, in actual practice, the function or functionsperformed by that block may be performed in a single component or acrossmultiple components, and/or may be performed using hardware, usingsoftware, or using a combination of hardware and software. To clearlyillustrate this interchangeability of hardware and software, variousillustrative components, blocks, modules, circuits, and steps aredescribed below generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present disclosure. Also, the example devices may includecomponents other than those shown, including well-known components suchas a processor, memory and the like.

Aspects of the present disclosure are applicable to any suitableelectronic device (such as a security system with one or more cameras,smartphones, tablets, laptop computers, digital video and/or stillcameras, web cameras, and so on) configured to or capable of capturingimages or video. While described below with respect to a device havingor coupled to one camera, aspects of the present disclosure areapplicable to devices having any number of cameras (including nocameras, where a separate device is used for capturing images or videowhich are provided to the device), and are therefore not limited todevices having one camera. Aspects of the present disclosure areapplicable for capturing still images as well as for capturing video,and may be implemented in devices having or coupled to cameras ofdifferent capabilities (such as a video camera or a still image camera).

The term “device” is not limited to one or a specific number of physicalobjects (such as one smartphone, one camera controller, one processingsystem and so on). As used herein, a device may be any electronic devicewith one or more parts that may implement at least some portions of thisdisclosure. While the below description and examples use the term“device” to describe various aspects of this disclosure, the term“device” is not limited to a specific configuration, type, or number ofobjects.

For some AF operations for a camera, the position of a camera lens maybe automatically determined to place a portion of the scene in focus forthe camera. In one example, a device may display a preview of the scenefrom the camera and a user may touch or otherwise indicate an area ofthe preview to indicate the portion of the scene to be in focus for thecamera. In another example, the device may automatically determine aportion of the scene to be in focus. In response, the device may performan AF operation to adjust the camera lens position so that the portionof the scene is in focus for subsequent image captures by the camera.

FIG. 1 is a depiction of an example scene 100 to be captured by acamera. As shown, different portions of the scene are at differentdistances or depths from the camera. If a portion of the scene at onedepth is to be in focus for the camera, other portions of the scene maybe out of focus in an image provided by the camera. For example, if thepier near the camera is to be in focus, the boat on the right, theremaining portions of the pier, and the buildings and coast toward thehorizon may be out of focus.

A device may trigger an AF operation if the device senses or determinesthat the scene has changed. In one example of scene change, the cameramay move so that the entire scene in the camera's field of capturechanges (called a global motion). In a further example of scene change,an object may move in the scene or obstruct a portion of the scene(called a local motion). Examples of local motion include a personwalking in front of the camera, a user's finger covering a portion ofthe camera lens or camera aperture to the camera sensor, and so on.

However, some local motions may not impact a region of interest (ROI) orregion that is of interest to a user. For example, referring back toFIG. 1, the ROI may be the portion of the pier closest to the camera. Asa result, the focal length for the camera may be set for the ROI. If afinger or object covers the top left portion of a camera sensor (even ifnot covering the portion of the sensor corresponding to the ROI), thedevice may determine that the scene has changed and retrigger an AFoperation. As a result, the focal length may again be determined andset. While the focal length is being determined and set, a user may beprevented from using the camera to capture an image. Additionally, auser may notice the retriggered AF operation from a preview. Forexample, an AF operation may include changing the focal length to searchfor a final focal length. In this manner, a portion of the scene in apreview stream may go in and out of focus during the AF operation, orthe preview may be otherwise temporarily interrupted. Unnecessarilytriggering an AF operation may therefore negatively impact a userexperience.

Types of AF operations include contrast detection (CD) and phasedifference (PD) (CDAF and PDAF, respectively). For PDAF, a camera sensormay include photodiodes distributed across the camera sensor, with eachphotodiode being used to measure an intensity of the light received bythe photodiode. The light intensity may be measured as a luminance oranother means of measuring brightness. One example camera sensor is adual photodiode (2PD) sensor, where each pixel of the camera sensorincludes two photodiodes. Other example camera sensors are sensors witha sparser distribution of photodiodes than a 2PD sensor. For example, a2×1 sensor may include a photodiode for every 2×2 patch of pixels of thecamera sensor, a 4×1 sensor may include a photodiode for every 4×4 patchof pixels of the camera sensor, and so on.

In performing PDAF, two instances of the light emitted from an object inthe scene being captured passes through different portions of a cameralens of a camera. The camera lens refracts the instances of light to thecamera sensor, and the camera sensor senses or receives the refractedlight. If the two instances of light align on the camera sensor afterpassing through the camera lens, the scene is determined to be in focusfor image capture. If the two instances hit the camera sensor atdifferent locations, the scene is determined to be out of focus. Forexample, the distribution of photodiodes is used to measure theluminance's of the received light, and the measured luminances areanalyzed to determine where the two instances hit the camera sensor.

FIG. 2A is a depiction of a camera 201 with a camera lens 202 at a focallength 208A (from the camera sensor 204) so that an object 206 is infocus at focus distance 210. FIG. 2B is a depiction of the camera 201with the camera lens 202 at too long of a focal length 208B so that anobject 206 is out of focus. FIG. 2C is a depiction of the camera 201with the camera lens 202 at too short of a focal length 208C so that anobject 206 is out of focus. The illustrated bumps indicate thedistribution of measured luminance of the instances of light refractedby the camera lens 202 and received by the photodiodes located atdifferent places of the camera sensor 204. As shown, the camera lens 202refracts the instances of light from the object 206 to the camera sensor204. If the distributions of measured luminance (illustrated as bumps)do not align, the object 206 is out of focus for images captured usingthe focal length (such as in FIG. 2B and FIG. 2C). If the distributionsof measured luminance align, the object 206 is in focus for imagescaptured using the focal length (such as in FIG. 2A). In some exampleimplementations, the camera includes an actuator (not shown) to move thecamera lens 202 toward or away from the camera sensor 204, thusadjusting the focal length (and therefore the focus distance).

The PD may be a distance in camera sensor pixels between the twoinstances hitting the camera sensor 204. In FIG. 2A, the PD is zerobecause the two instances align on the camera sensor 204. In FIGS. 2Band 2C, the PD is greater than and less than zero, respectively, sincethe two instances do not align on the camera sensor 204. The focallength is linearly correlated to the PD.

FIG. 3 is a depiction 300 of an example correlation between focal lengthand PD for PDAF. The correlation between the PD and focal length may beknown, such as the rate of change (slope of the line). A final focallength 302 is the focal length corresponding to the measured PD beingzero. The camera lens position may be adjusted from the initial orcurrent focal length 304 toward the final focal length 302 to place anobject or portion of the scene in focus. If the PD 306 is measured foran initial focal length 304, the focal length difference 308 from thefinal focal length 302 may be determined from the known rate of change(slope of the line) and the initial focal length 304 (corresponding tothe offset of the line). The camera lens position may then be adjustedapproximately by the focal length difference 308.

While the above examples for PDAF are regarding photodiodes dispersedacross the camera sensor, other ways for performing PDAF may be used.For example, a mirror may be used to reflect the refracted light towarda camera sensor. Holes in the mirror may allow some light to passthrough to a separate light sensor. The separate light sensor may be anarray of photodiodes, a one-dimensional or two-dimensional strip ofphotosensitive pixels (such as in a charge-coupled device), and so on.The separate light sensor is used in measuring a PD, and the camerasensor is used in capturing an image. Therefore, the present disclosureshould not be limited to a specific example of PDAF.

Another type of AF operation is CDAF. For CDAF, a contrast is determinedusing the camera sensor without using a separate light sensor orphotodiodes. The contrast is a difference in measured light intensity(such as luminance) between neighboring pixels of the camera sensor.

FIG. 4A is a depiction of a camera 401 with a camera lens 402 at a focallength 408A (from the camera sensor 404) so that an object 406 is infocus at focus distance 410. FIG. 4B is a depiction of the camera 401with the camera lens 402 at too long of a focal length 408B so that anobject 406 is out of focus. FIG. 4C is a depiction of the camera 401with the camera lens 402 at too short of a focal length 408C so that anobject 406 is out of focus. The illustrated bumps indicate thedistribution of measured luminance of light from the object 406 andrefracted by the camera lens 402 toward the camera sensor 404. An object406 is considered in focus when the distribution is tighter (such as asmaller standard deviation) than the distributions for the other focallengths. As shown, the object 406 is out of focus when the distributionof measured light intensity for the object is spread out compared to thedistribution for the focal length when the object is in focus. In thismanner, the difference in light intensity for light from the object 406received by neighboring pixels of the camera sensor 404 is greatest whenthe object 406 is in focus (the slope of the curve is greatest when theobject 406 is in focus). For example, when an object is out of focus, animage of the object appears blurry, and the difference in lightintensity between pixels (contrast) is less than when the object is infocus. While contrast is described as a difference in light intensitybetween neighboring pixels, contrast may be a difference in chrominanceor a difference in some combination of chrominance and light intensity,and the present disclosure should not be limited to specific examples ofcontrast or CDAF.

FIG. 5 is a depiction 500 of an example correlation between focal lengthand contrast for CDAF. As shown, the correlation between focal lengthand contrast is parabolic/second order. The exact curvature may differ,and the depiction 500 is for illustrative purposes only. For example,the correlation may be expressed in general by a second order equationy=ax²+bx+c, where the contrast is y, the focal length is x, thecurvature of the parabola is indicated by a, the slope of the parabolais indicated by b, and the offset of the parabola is indicated by c. Thefinal focal length 502 for which the object is in focus corresponds tothe contrast being at a maximum compared to the contrasts for otherfocal lengths (the vertex of the parabola). For example, the vertex maybe −b/2a for the above second order equation.

For CDAF, the focal length is adjusted and the contrast determined untilthe final focal length 502 is found. For example, one or more coarseadjustments 506 to the focal length from the initial focal length 504may be performed until reaching a focal length within a threshold of thefinal focal length 502. A coarse adjustment 506 is an adjustment to thefocal length of the camera that is greater than a fine adjustment 508.Coarse adjustments and fine adjustments may be fixed or variable insize, pre-configured or configurable, or any other suitable types offocal length adjustments for CDAF, and the present disclosure should notbe limited to a specific example of adjustments to the focal length.

In searching for or converging to the final focal length 502, a coarseadjustment 506 may be performed if the difference between a previouscontrast and a current contrast is increasing or steady to previousdistances and/or the difference is greater than a threshold. Conversely,the difference between the previous contrast and the current contrastdecreasing and/or the difference being less than a threshold mayindicate that the current focal length is within a threshold of thefinal focal length, and another coarse adjustment 506 may not beperformed. Then, one or more fine adjustments 508 may be performed untilconverging to the final focal length 502. As a result, CDAF is aniterative process of measuring the contrast, adjusting the focal length,and again measuring the contrast until a final focal length 502 isdetermined.

The present disclosure uses the term “focus value” to include contrastand PD. For example, determining a focus value to place an object infocus may include determining a highest or greatest contrast ordetermining a PD closest to zero (which may be considered a “lowest PD”or a “smallest PD”). A focus value may also be any other suitabledetermination or measurement in determining a focal length or whether anobject is in focus, and the present disclosure should not be limited toa specific example of a focus value.

After a focal length is determined for a camera (such as by PDAF orCDAF), a device may retrigger an AF operation if the device determinesthat the scene in the camera's field of capture has changed. A “focalregion” for a camera is a portion of the scene as captured by the cameraand to be used in determining a focus value. The focal region may be aportion of the camera sensor, which may include a number of sensorpixels and be static or defined by the camera. A camera or camera sensormay include multiple focal regions. In determining that a scene haschanged, the device may measure a focus value for each focal region. Forexample, FIG. 6 is a depiction of an example image of the scene 600(such as scene 100 in FIG. 1). The device may determine a focus valuefor focal regions of the camera sensor or a light sensor that capturethe portions 602 of scene 600. For CDAF, the information from the camerasensor pixels corresponding to a portion 602 may be used to determine afocus value for the corresponding focal region. For PDAF, theinformation from the photodiodes corresponding to the focal region ofthe camera sensor capturing the portion 602 may be used to determine afocus value. As shown, the portions 602 and corresponding focal regionsmay be rectangles evenly distributed across the camera sensor or lightsensor. However, the portions and/or focal regions may be of anydimension, number, and distribution.

In some implementations of determining whether a scene is changing so totrigger an AF operation, the device may determine if a threshold numberof portions 602 are changing. For example, the device may store areference focus value for each focal region corresponding to theportions 602. The reference focus value may be the focus valuedetermined for the focal region when the focal length was last set (suchas during the previous AF operation). In determining whether a focalregion changes, the device may compare a difference between a currentfocus value and the reference focus value to a threshold (which may beany suitable value, including zero). The device may determine that thefocal region changes when the difference is greater than the threshold.

In another implementation of determining a scene change, the device maydetermine whether an overall focus measurement or value for an image ischanging between images. The device may determine an overall focus valuefrom the focus values determined for each of the focal regions (such asthe focal regions associated with portions 602 in FIG. 6). For example,the overall focus value for the image may be an average focus value, asummation of the focus values, a median focus value, and so on. Thedevice may then compare the determined overall focus value for thecurrent image to the overall focus value for a previous image (such as areference overall focus value). The previous image may be the directlyprevious image received by the device (or captured by the camera) oranother previous image (such as the image received by the device orcaptured by the camera when the focal length was last set for thecamera). If the difference between overall focus values is greater thana threshold, the device may determine that the scene has changed totrigger an AF operation.

While the examples describe comparing overall focus values for differentimages, images may not need to be captured to determine an overall focusvalue. For example, the photodiodes or light sensor for PDAF may operateindependently of the camera sensor. As a result, the photodiodes orlight sensor may measure intensities without the camera sensor capturingan image. Therefore, the measuring focus values or overall focus valuesbetween images may also include measuring focus values that aredetermined without capturing corresponding images.

The ROI or region of importance to a user may not be affected by thescene change. For example, one or more portions 602 of the scene may beaffected by an object blocking a portion of the camera sensor (such as afinger covering a corner, an object entering the edge of the field ofcapture for the camera, and so on), but the portion of the camera sensorbeing blocked or affected may not correspond to the portions 602associated with an ROI.

For example, FIG. 7 is a depiction of an example image 700 of the scene100 in FIG. 1 with a portion 702 of the scene blocked. The portion 702being blocked may be as a result of a finger covering a portion of thecamera lens or an object entering the field of capture for the camera.The portions 602 that are highlighted are affected, and a device maydetermine that the highlighted portions are changed (such as describedabove). If the number of highlighted portions is greater than athreshold, the device may determine to retrigger an AF operation todetermine a focal length for the camera. Alternatively, if an overallfocus value changes more than a threshold from a previous overall focusvalue, the device may determine to retrigger an AF operation todetermine a focal length for the camera. If the pier closest to thecamera is the ROI, the highlighted portions may not be the portionsassociated with the ROI. FIG. 8 is a depiction of the example image 700and illustrating the portions 602 corresponding to an example ROI of thescene (the pier closest to the camera). As shown, the portion 702 of thescene blocked does not affect the portions 602 corresponding to theexample ROI in the scene.

In some example implementations, a device may exclude one or more focalregions (where each focal region may correspond to, e.g., one of theportions 602) for use in determining whether to trigger an AF operation.Additionally or alternatively, the device may exclude one or more focalregions for use in performing an AF operation (such as determining afocal length). In determining which focal regions to exclude, the devicemay generate a depth map. For example, the device may generate a depthmap of focal lengths for the focal regions corresponding to portions602.

FIG. 9 is a block diagram of an example device 900 for determining whento perform an AF operation, performing an AF operation, and/orgenerating a depth map. The example device 900 may include or be coupledto a camera 902, a processor 904, a memory 906 storing instructions 908,and a camera controller 910. The device 900 may optionally include (orbe coupled to) a display 914 and a number of input/output (I/O)components 916. The device 900 may include additional features orcomponents not shown. For example, a wireless interface, which mayinclude a number of transceivers and a baseband processor, may beincluded for a wireless communication device. The device 900 may includeor be coupled to additional cameras other than the camera 902. Thedisclosure should not be limited to any specific examples orillustrations, including the example device 900.

The camera 902 may be an example of the camera 201 in FIGS. 2A-2C or thecamera 401 in FIGS. 4A-4C. The camera 902 may be capable of capturingindividual image frames (such as still images) and/or capturing video(such as a succession of captured image frames). The camera 902 mayinclude a single camera sensor, or be a dual camera module or any othersuitable module with multiple camera sensors. The camera 902 may alsoinclude a separate light sensor or multiple photodiodes as part of thecamera sensor. The memory 906 may be a non-transient or non-transitorycomputer readable medium storing computer-executable instructions 908 toperform all or a portion of one or more operations described in thisdisclosure. The device 900 may also include a power supply 918, whichmay be coupled to or integrated into the device 900.

The processor 904 may be one or more suitable processors capable ofexecuting scripts or instructions of one or more software programs (suchas instructions 908) stored within the memory 906. In some aspects, theprocessor 904 may be one or more general purpose processors that executeinstructions 908 to cause the device 900 to perform any number offunctions or operations. In additional or alternative aspects, theprocessor 904 may include integrated circuits or other hardware toperform functions or operations without the use of software. While shownto be coupled to each other via the processor 904 in the example of FIG.9, the processor 904, the memory 906, the camera controller 910, theoptional display 914, and the optional I/O components 916 may be coupledto one another in various arrangements. For example, the processor 904,the memory 906, the camera controller 910, the optional display 914,and/or the optional I/O components 916 may be coupled to each other viaone or more local buses (not shown for simplicity).

The display 914 may be any suitable display or screen allowing for userinteraction and/or to present items (such as captured images, video, ora preview image) for viewing by a user. In some aspects, the display 914may be a touch-sensitive display. The I/O components 916 may be orinclude any suitable mechanism, interface, or device to receive input(such as commands) from the user and to provide output to the user. Forexample, the I/O components 916 may include (but are not limited to) agraphical user interface, keyboard, mouse, microphone and speakers, andso on. The display 914 and/or the I/O components 916 may provide apreview image to a user and/or receive a user input for adjusting one ormore settings of the camera 902 (such as selecting and/or deselecting aROI of the scene by touching a portion of a displayed preview image).

The camera controller 910 may include an image signal processor 912,which may be one or more image signal processors to process capturedimage frames or video provided by the camera 902. In some exampleimplementations, the camera controller 910 (such as the image signalprocessor 912) may also control operation of the camera 902. In someaspects, the image signal processor 912 may execute instructions from amemory (such as instructions 908 from the memory 906 or instructionsstored in a separate memory coupled to the image signal processor 912).In other aspects, the image signal processor 912 may include specifichardware. The image signal processor 912 may alternatively oradditionally include a combination of specific hardware and the abilityto execute software instructions.

As described above, the device 900 may use multiple focal regions (suchas corresponding to portions 602 in FIG. 6) for use in determiningwhether to trigger an AF operation or for use in performing an AFoperation for the camera 902. In some example implementations, thedevice 900 may exclude one or more focal regions when the portions ofthe scene corresponding to the focal regions (such as portions 602 inFIG. 6) are at different depths in the scene than a ROI or region ofimportance to a user. FIG. 10 is an illustrative flow chart depicting anexample operation 1000 for excluding one or more focal regions (such ascorresponding to one or more portions 602 in FIG. 6) for use intriggering and/or performing an AF operation for a camera 902. While thefollowing examples are described in relation to the example device 900and the camera 902, the example methods may be performed by devices orsystems of other configurations and capabilities. For example, thecamera 902 may be separate and communicably coupled to the device 900(such as a security system with multiple cameras communicably coupled toa hub), the device 900 may be a smartphone or other electronic device,the device 900 may be a vehicle including one or more cameras fordetecting scene changes, and so on. The following examples are forillustrative purposes, and the present disclosure should not be limitedto any specific device (including device 900) or ways for performingaspects of the present disclosure.

Beginning at 1002, the device 900 may receive an image stream (includinga sequence of images) of a scene from the camera 902. For example, thecamera 902 may capture an image stream and provide the image stream tothe camera controller 910. The device 900 may use the image stream andfocal regions corresponding to portions of the image stream to determineone or more focal regions to be excluded from being used in determiningto perform an AF operation or for performing an AF operation for thecamera 902 (1004). For example, the device 900 may determine a ROI forthe scene in the camera's field of capture (such as determining a ROIfor a portion of the scene indicated by a user input, determining a ROIfor an identified face in the scene, and so on). The device 900 may thendetermine to exclude any focal regions not associated with the ROI. Inone example, a focal region may not be associated with the ROI if noportion of the focal region overlaps with any portion of the ROI. Inanother example, a focal region may not be associated with the ROI ifthe scene contents for the focal region are at a different depth thanthe scene contents for the ROI. The device 900 may then exclude thedetermined one or more focal regions not associated with the ROI frombeing used in triggering or performing an AF operation (1006). Forexample, the device 900 may exclude focal regions corresponding to asubset of the portions 602 in FIG. 6 that are not associated with a ROIof the pier closest to the camera. The focal regions not excludedcorrespond to the remaining portions 602 in FIG. 8.

After one or more focal regions are excluded, the device 900 may use theremaining focal regions (the focal regions not excluded) in determiningwhether to perform/trigger an AF operation or for performing an AFoperation for the camera 902 (1008). In some examples, the device 900may determine whether a threshold of remaining focal regions is changingduring a scene change (such as objects moving, blocking a portion of thecamera sensor, or a global motion of the camera). If the threshold isreached, the device 900 may determine to trigger an AF operation. Insome example implementations, since the remaining focal regions areassociated with the ROI, the threshold may be lower than a thresholdused for triggering an AF operation when no focal regions are excluded.In one example, the threshold may be one focal region, five percent offocal regions, or another suitable amount. In another example, if thethreshold for triggering an AF operation is a ratio of the number offocal regions changing to the number of focal regions that are stable(not changing), the threshold for the remaining focal regions associatedwith the ROI may be the same ratio of the number of remaining focalregions changing to the number of remaining focal regions that arestable. As described above, a focal region may be changing if the focalregion's associated focus value at different times differ by more than athreshold. A focal region may be stable if the difference between thefocal region's associated focus values remains within a threshold.

In some other examples, the device 900 may determine an overall focusvalue from the focus values for the remaining focal regions (such as anaverage focus value, a median focus value, a summation of focus values,and so on). As a result, the excluded focal regions may not be used indetermining the overall focus value. The device 900 may then compare theoverall focus value to a previously determined overall focus value todetermine if the difference between overall focus values is greater thana threshold. In some example implementations, the previously determinedoverall focus value may be a reference overall focus value, which may bean overall focus value determined when the focal length was last set forthe camera 902 during a previous AF operation.

Referring back to FIG. 6 and FIG. 8 to illustrate determining to triggeran AF operation, since the object blocking the portion 702 of the scenedoes not affect the remaining portions 602 in FIG. 8, the number offocal regions (associated with portions 602 in FIG. 8) that are changingis zero (and does not reach a threshold for determining to trigger an AFoperation). As a result, the device 900 may determine not to trigger anAF operation to adjust the focal length as a result of the blockage ofthe portion 702 of the scene.

In some example implementations of determining one or more focal regionsto be excluded, the device 900 may determine a ROI and determine onlythe focal regions associated with the ROI not to be excluded. In someother example implementations, the device 900 may also determine not toexclude one or more focal regions surrounding the focal regionsassociated with the ROI. For example, if a center group of focal regionsis associated with a ROI, the device 900 may determine not to exclude alayer of one focal region surrounding the center group of focal regions.As a result, the device 900 may be able to predict if an object/localmotion is about to affect the center group of focal regions if one ormore focal regions of the layer surrounding the center group areaffected by the object.

Different portions of a scene may be in focus at the same focal length.For example, referring back to FIG. 6, the boat in the right side of thefield of capture for the scene 600 and the pier on the left side of thefield of capture for the scene may be in focus at the same focal lengthsince they are similar depths from the camera. In some exampleimplementations, the device 900 may determine which focal regions whosecorresponding portion of the scene is in focus when the ROI is in focus.Also, the device 900 may determine whether any of the focal regions withcorresponding scene portions overlapping at least a portion of the ROIhas a different focus value than the other focal regions withcorresponding scene portions overlapping at least a portion of the ROI.For example, the illustrated portions 602 in FIG. 8 may not include thewater between the pier's post and the pier's floor on the bottom leftand bottom right of the scene 600. As a result, the focal regionscorresponding to the portions primarily including the water may beexcluded.

In this manner, contiguous or non-contiguous groups of focal regions maycorrespond with different portions of the scene that is in focus at thesame focal length. The device 900 may determine to exclude the focalregions whose corresponding portions of the scene are not in focus atthe focal length for which the ROI is in focus. To determine one or morefocal regions to be excluded from being used, the device 900 maygenerate and use a depth map for the focal regions, with some exampleimplementations of a depth map described below. The depth map mayprovide an indication of a focal length for each focal region that thecorresponding portion of the scene is in focus.

FIG. 11 is an illustrative flow chart depicting an example operation1100 for generating a depth map. In some example implementations,generating a depth map may be included in determining one or more focalregions to be excluded from being used in determining to perform an AFoperation or for performing an AF operation (such as 1004 in FIG. 10).For example, the determined values in the depth map may be determinedfocus values or focal lengths of the camera for when a correspondingfocal region is in focus. The device 900 may use the determined valuesin the depth map to exclude one or more focal regions (such as 1006 inFIG. 10). For example, a focal region may be excluded if the determinedfocus value or focal length is outside a predetermined distance from areference focus value or focal length (such as described below regardingFIG. 18). In another example, a focal region may be excluded if thedetermined focus value or focal length is outside a predetermineddistance from a focus value or focal length for a focal regiondetermined to be used in triggering or performing an AF operation of thecamera (such as described below regarding FIG. 17). The remaining(non-excluded) focal regions may then be used in determining when toperform an AF operation or in performing an AF operation.

Beginning at 1102, the device 900 may set the camera lens of the camera902 to an initial focal length. In some example implementations, theinitial focal length may be the smallest focal length or the largestfocal length for the camera 902. In some other example implementations,the initial focal length may be the last used focal length or any othersuitable focal length. After setting the initial focal length for thecamera 902, the device 900 may measure intensities from the scene usingthe initial focal length (1104). The intensities may be lightintensities, chrominance intensities, or another measured intensity. Inone example, the device 900 may receive an image captured by a camerasensor, and the device 900 may use the image values (such as RGB or YUVvalues) at one or more points or one or more regions of the image todetermine intensities in the scene captured in the image (such asdetermining one or more light intensities in the scene). In anotherexample, the device 900 may use photodiodes to measure lightintensities.

The device 900 may then determine from the measurements a focus valuefor each focal region (1106). For example, the device 900 may determinea PD for each focal region if using PDAF (such as from photodiodemeasurements), or the device 900 may determine a contrast for each focalregion if using CDAF (such as from an image capture by the image sensorof the camera 902). FIG. 12 is a depiction 1200 of focus values 1202determined for the focal regions. For example, the depiction may be foran image of the scene 600 at an initial focal length, with the focusvalues 1202 of the focal regions corresponding to portions 602. If theinitial focal length is set to the smallest focal length for the camera902, objects at further depths from the camera 902 may appear more outof focus than objects at closer depths from the camera 902. The focusvalues 1202 depict an example contrast determined for each focal regionfor the scene 600 in FIG. 6. The shading for the focus valuescorresponds to the magnitude of a contrast (with a darker shadingcorresponding to a lower contrast). For example, focus values 1204(which may correspond to the sky) are lower than focus values 1206(which may correspond to the farthest end of the pier and the boats inthe scene 600). Focus values 1206 are lower than focus values 1208(which may correspond to the water on either side of the pier in thescene 600), and focus values 1208 are lower than focus values 1210(which may correspond to the portion of the pier closest to the camera902 and/or may be the ROI).

A higher contrast corresponds to the corresponding portion of the scenebeing closer to in focus. Therefore, focus values 1210 correspond toportions of the scene (such as the pier closest to the camera 902 inscene 600) being closer to in focus than portions of the scenecorresponding to focus values 1204-1208. In some exampleimplementations, the device 900 may normalize the focus values. Forexample, the contrasts may be normalized to a scale of 0-10, 0-9, or1-10 for the focus values in the depiction 1200. In some other exampleimplementations, the device 900 may use the focus values withoutnormalization. Otherwise, the illustrated focus values may be anotherindication of the actual focus values determined. If a focus value is aPD instead of a contrast, a lower focus value (a focus value closer tozero) corresponds to the corresponding portion of the scene being closerto in focus than if the focus value is higher (a focus value furtherfrom zero). As a result, the change in value from focus values 1204 tofocus values 1210 may decrease for PD instead of increase for contrast.Normalizing a PD may include normalizing a magnitude of the PD.Otherwise, the values may be another indication of the PD determined bythe device 900 for each focal region.

Referring back to FIG. 11, the device 900 may determine if another focallength is to be set for the camera (1108). As described above, a device900 performing CDAF may repeatedly adjust the focal length (such asperforming coarse and fine adjustments) and measure the contrast until afinal focal length is determined. Similarly, the device 900 may adjustthe focal length and again determine the contrast for each focal region.In one example, the device 900 may step the camera lens through eachfocal length, determining contrasts for the focal regions at each focallength. In another example, the device 900 may adjust the focal length afixed number of steps greater than one, and a focus value for focallengths being skipped may be interpolated from the determined focusvalues. In a further example, the device 900 may vary the steps ofadjusting the focal length, such as using larger steps for the edges ofthe range of focal lengths than for the middle of the range of focallengths for the camera 902. Additionally, the camera lens may movethrough the focal lengths in one direction (such as from largest focallength to smallest focal length or smallest focal length to largestfocal length), or the camera lens may move in either direction. Othersuitable means for adjusting the focal length may be used, and thepresent disclosure should not be limited to any specific example foradjusting the focal length.

If the device 900 is to adjust the camera lens to another focal length(1108), the device 900 sets the camera lens to the next focal length(1110) and measures light intensities from the scene using the new focallength (with the process reverting to 1104). As a result, the device 900may determine for a focal region multiple focus values, with each focusvalue associated with a different focal length. FIG. 13 is a depiction1300 of focus values 1302 for the focal regions at an adjusted focallength. For example, if the focal length is increased by a step from thesmallest focal length for the camera 902, the scene (or at least aportion of the scene) as captured in an image by the camera sensor forthe adjusted focal length may appear closer to in focus than the sceneas captured in a previous image for the initial focal length. As shown,the contrast may still increase from focus values 1304 through focusvalues 1306 and 1308 to focus values 1310.

When recursively adjusting the focal length and determining the focusvalues, the focus values 1310 may begin to decrease at some focallength. For example, the portions of the scene (such as portions 602 inFIG. 6) corresponding to the focus values 1308 may be closer to in-focus(such as higher corresponding contrasts or lower corresponding PDs) thanthe portions of the scene 600 corresponding to the focus values 1310.FIG. 14 is a depiction 1400 of focus values 1402 at another adjustedfocal length. For the focal length, the water to the left and right ofthe pier and the intermediate portion of the pier in the scene may becloser to in focus (such as higher contrasts or lower PDs) than theportion of the pier closest to the camera 902. Further, the portions ofthe scene corresponding to focus values 1306 and focus values 1310 inFIG. 13 (such as the end of the pier and the portion of the pier closestto the camera 902) may have similar focus values 1406 in FIG. 14, whichis greater than the focus values 1404 (corresponding to the sky) butless than the focus values 1408 (corresponding to the water to the rightand left of the pier and the intermediate portion of the pier).

Referring back to FIG. 11, if no other focal length is to be set for thecamera 902 (1108), the device 900 may determine for each focal region afocal length for which the associated portion of the scene is in focus(1112). For example, the device 900 may set the camera lens from theshortest focal length to the longest focal length. Once the camera lensis adjusted to the longest focal length, the device 900 may not furtheradjust the focal length and determine for each focal region the focallength for which the associated portion of the scene is in focus.Alternatively, the device 900 may adjust the focal length any suitablenumber of times and in any suitable fashion. FIGS. 12-14 depict examplefocus values (which may be normalized or an indication of the focusvalues determined by the device 900) for the focal regions correspondingto the portions of the scene (such as corresponding to the portions 602of scene 600 in FIG. 6) at different focal lengths. If the device 900determines a contrast as the focus value, the device 900 determines foreach focal region the focal length at which the contrast is the highest.If the device 900 determines a PD as the focus value, the device 900determines for each focal region the focal length at which the PD islowest (closest to zero). The group of example focus values (or exampleindications of focus values) in FIGS. 12-14 may be considered an exampledepth map of focus values.

In some example implementations for determining the focal lengths forthe focal regions, the device 900 may determine and store an array offocus values for each focal region. FIG. 15 is a depiction 1500 of anarray generated for a first focal region. The focus values for the samefocal region across the different focal lengths may be included in thearray. For example, the depictions 1200, 1300, and 1400 of focus valuesmay be across an increasing focal length 1504 and included with otherfocus values determined at different focal lengths. If focal region ncorresponds to the bottom left focal region for the camera sensor, thearray includes 8 as the first value (the bottom left focus value indepiction 1200 of focus values), 9 as the second value (the bottom leftfocus value in depiction 1300 of focus values), and 5 as an intermediatevalue (the bottom left focus value in depiction 1400 of focus values).An array similar to array 1502 may be generated and stored for eachfocal region. In some example implementations, the arrays may be storedin the memory 906 of device 900 or in another storage included in orremote to the device 900.

The same position in the arrays corresponds to the same focal length.For example, the first position in each of the arrays may correspond toan initial focal length, the second position may correspond to the firstadjusted focal length, and so on until the last position corresponds tothe final adjusted focal length. In this manner, the device 900 maydetermine which array position includes the largest contrast or thesmallest PD, which corresponds to a focal length for which the portionof the scene corresponding to the focal region is closest to in focusfor the camera 902.

Referring back to FIG. 11, the device 900 may generate a depth map ofthe determined focal lengths for the focal regions (1114). The depth mapof focal lengths may instead include indications of the focal lengthsdetermined for the focal regions. The values in the depth map may be,e.g., an array position determined for each focal region, a determinedfocal length for each focal region, a normalized value (such as from1-10) indicating a determined focal length for each focal region, anumber of steps from an initial focal length, or any other suitableindication of the focal length. FIG. 16 is a depiction of an exampledepth map 1600 of normalized focal lengths for the focal regions forFIG. 12 through FIG. 14. For the example depth map 1600, the focallengths are normalized to a range of 1-10. As shown, the focal lengthsin the group 1602 may be similar, the focal lengths in the group 1604may be similar, the focal lengths in the groups 1606 may be similar, andthe focal lengths in the group of 1608 may be similar.

While FIGS. 11-15 illustrate that multiple focal lengths may be used,the device 900 alternatively may use one focal length to generate adepth map. For example, if the device 900 performs PDAF, a determined PDfor each focal region at one focal length may be used to determine afinal focal length for each focal region and generate the depth map fromthe final focal lengths. In the example, the device 900 may optionallygenerate a depth map of focus values and use the depth map of focusvalues to generate a depth map of focal lengths. In another example, thedevice 900 may still use multiple focal lengths even if determining PDsas focus values, and the present disclosure should not be limited to aspecific number of focal lengths to be used in generating a depth map.

The device 900 may use a generated depth map of focal lengths todetermine one or more focal regions to be excluded from use intriggering an AF operation or performing an AF operation. FIG. 17 is anillustrative flow chart depicting an example operation 1700 for using agenerated depth map of focal lengths to determine which focal regions toexclude. Example operation 1700 may be an example implementation of step1004 in FIG. 10.

Beginning at 1702, the device 900 may determine one or more primaryfocal regions. In some example implementations, a camera 902 focallength may be set to place contents at the center of the camera 902field of capture in focus, and the device 900 may determine one or morecenter focal regions to be the primary focal regions. In some otherexample implementations, the device 900 may identify one or more facesin a field of capture of the camera 902, and the device 900 maydetermine one or more focal regions corresponding to the location(s) ofthe one or more faces to be the primary focal regions. In some furtherexample implementations, the device 900 may receive a user input (suchas a user touching a portion of a preview on the display 914) toindicate a ROI, and the device 900 may determine one or more focalregions corresponding to the location in the scene of the user input.

After determining one or more primary focal regions (1702), the device900 may determine from the depth map whether any other focal regionshave a determined focal length similar to the focal length(s) for theone or more primary focal regions. In some example implementations, thedevice 900 may begin at a first focal region and determine if its focallength from the depth map is within a predetermined distance from thefocal length of the one or more primary focal regions (1704). In oneexample, the device 900 may determine if the determined focal lengthsare the same for the first focal region and the one or more primaryfocal regions. As a result, the tolerance in depths in the scene may beminimized in determining which focal regions to use for triggering orperforming an AF operation. If the predetermined distance is greaterthan zero, the tolerance in depths is greater than the tolerance indepths for the predetermined distance being zero. As a result, thenumber of focal regions to be excluded may decrease as the predetermineddistance increases.

In some example implementations, the device 900 may determine areference focal length from the determined focal lengths for the one ormore primary focal regions. The device 900 may then compare thereference focal length to the determined focal lengths in the depth map.

If the determined focal length for the focal region is not within thepredetermined distance from the focal length for the one or more primaryfocal regions (1706), the device 900 may determine to exclude the focalregion from being used in triggering an AF operation or performing an AFoperation (1708). If the determined focal length for the focal region iswithin the predetermined distance from the focal length for the one ormore primary focal regions (1706), the device 900 may determine to usethe focal region for triggering an AF operation or performing an AFoperation. As a result, the device 900 may not exclude the focal region(not shown), with the process continuing to 1710. In 1710, the device900 may determine if a next focal region's focal length from the depthmap is within a predetermined distance from the focal length of the oneor more primary focal regions (1710). Step 1710 may be performedsimilarly to 1704, as described above.

In some additional or alternative implementations, the device 900 maycompare a focal length of a primary focal region to the focal lengths ofother primary focal regions (not shown) to determine whether to excludeone of the primary focal regions. For example, if the device 900determines four focal regions corresponding to the scene of capture forthe camera 902 to be the primary focal regions, the device 900 maydetermine if a focal length of one of the four focal regions is anoutlier compared to the other three focal lengths (such as if the focallength for one primary region is more than the predetermined distanceaway from the focal lengths for the other three focal regions). As such,one or more primary focal regions may be excluded in addition oralternative to one or more non-primary focal regions being excluded.

In some additional example implementations, instead of determining oneor more primary focal regions, the device 900 may compare a differencebetween the final focal length determined from the previous AF operationand the determined focal lengths in the depth map. The device 900 maystore the final focal length for the camera 902 from a previous AFoperation. For example, if the device 900 is completing initializationof the camera 902, the focal length of the camera 902 determined duringthe AF operation during initialization may be stored. The stored focallength or the focal length determined from the previous AF operation maybe considered the reference focal length.

FIG. 18 is an illustrative flow chart depicting an example operation1800 for determining which focal regions to exclude. Beginning at 1802,the device 900 may compare a determined focal length for a first focalregion to the reference focal length. For example, the determined focallength may be the focal length at which the portion of the scenecorresponding to the first focal region is in focus (such as the focallength corresponding to the highest contrast or the PD closest to zeroas compared to the other focal lengths). In some exampleimplementations, the focal length for the first focal region may be thefirst focal region's focal length indicated by a generated depth map offocal lengths for the focal regions.

If the difference between the focal length for the focal region and thereference focal length is greater than a threshold (1804), the device900 may determine to exclude the focal region from being used todetermine whether to trigger an AF operation or to perform an AFoperation (1806). The device 900 may then compare a determined focallength for a next focal region to the reference focal length (1808), andthe process reverts to decision block 1804. If the difference betweenthe focal length for the focal region and the reference focal length isless than a threshold (1804), the device 900 may not exclude the focalregion from being used to determine when to trigger an AF operation orto perform an AF operation. In this manner, the process may skip to1808, and the device 900 may compare a determined focal length for anext focal region to the reference focal length. Steps 1804-1808 may berepeated for each of the focal regions, and the device 900 thusdetermines which focal regions to exclude.

When the device 900 determines the one or more focal regions that are tobe excluded from use in triggering an AF operation or performing an AFoperation, the device 900 may exclude the determined focal regions. Insome example implementations, the device 900 may use the remaining focalregions to determine an overall focus value to be used in determiningwhether to trigger an AF operation. In some other exampleimplementations, the device 900 may use the remaining focal regions toperform an AF operation (such as CDAF or PDAF) to determine a new focallength for the camera 902.

The device 900 may determine to exclude one or more focal regions orgenerate a depth map of determined focal lengths at any frequency. Forexample, if a scene changes and then remains stable for a thresholdnumber of image frames from the camera 902, the device 900 may againdetermine which of all of the focal regions to exclude from use infurther determining whether to trigger an AF operation or in performingan AF operation.

The techniques described herein may be implemented in hardware,software, firmware, or any combination thereof, unless specificallydescribed as being implemented in a specific manner. Any featuresdescribed as modules or components may also be implemented together inan integrated logic device or separately as discrete but interoperablelogic devices. If implemented in software, the techniques may berealized at least in part by a non-transitory processor-readable storagemedium (such as the memory 906 in the example device 900 of FIG. 9)comprising instructions 908 that, when executed by the processor 904 (orthe camera controller 910 or the image signal processor 912), cause thedevice 900 to perform one or more of the methods described above. Thenon-transitory processor-readable data storage medium may form part of acomputer program product, which may include packaging materials.

The non-transitory processor-readable storage medium may comprise randomaccess memory (RAM) such as synchronous dynamic random access memory(SDRAM), read only memory (ROM), non-volatile random access memory(NVRAM), electrically erasable programmable read-only memory (EEPROM),FLASH memory, other known storage media, and the like. The techniquesadditionally, or alternatively, may be realized at least in part by aprocessor-readable communication medium that carries or communicatescode in the form of instructions or data structures and that can beaccessed, read, and/or executed by a computer or other processor.

The various illustrative logical blocks, modules, circuits andinstructions described in connection with the embodiments disclosedherein may be executed by one or more processors, such as the processor904 or the image signal processor 912 in the example device 900 of FIG.9. Such processor(s) may include but are not limited to one or moredigital signal processors (DSPs), general purpose microprocessors,application specific integrated circuits (ASICs), application specificinstruction set processors (ASIPs), field programmable gate arrays(FPGAs), or other equivalent integrated or discrete logic circuitry. Theterm “processor,” as used herein may refer to any of the foregoingstructures or any other structure suitable for implementation of thetechniques described herein. In addition, in some aspects, thefunctionality described herein may be provided within dedicated softwaremodules or hardware modules configured as described herein. Also, thetechniques could be fully implemented in one or more circuits or logicelements. A general purpose processor may be a microprocessor, and/orany conventional processor, controller, microcontroller, or statemachine. A processor may also be implemented as a combination ofcomputing devices, e.g., a combination of a DSP and a microprocessor,multiple microprocessors, one or more microprocessors in conjunctionwith a DSP core, or any other such configuration.

While the present disclosure shows illustrative aspects, it should benoted that various changes and modifications could be made hereinwithout departing from the scope of the appended claims. Additionally,the functions, steps or actions of the method claims in accordance withaspects described herein need not be performed in any particular orderunless expressly stated otherwise. For example, the steps of thedescribed example operations, if performed by the device 900, the cameracontroller 910, the processor 904, and/or the image signal processor912, may be performed in any order and at any frequency (such as steps1704-1710 in FIG. 17 or steps 1802-1808 in FIG. 18 may be performed inany order of focal regions and/or concurrently for multiple focalregions). Furthermore, although elements may be described or claimed inthe singular, the plural is contemplated unless limitation to thesingular is explicitly stated. Accordingly, the disclosure is notlimited to the illustrated examples and any means for performing thefunctionality described herein are included in aspects of thedisclosure.

What is claimed is:
 1. A method for a device configured to perform anautofocus (AF) operation for a camera, comprising: determining a focusvalue for each of a plurality of focal regions for a camera set to afocal length; determining, based on the focus values, one or more focalregions of the plurality of focal regions to exclude from use intriggering or performing an AF operation; and determining whether totrigger the AF operation, comprising: determining a second focus valuefor each of one or more of the plurality focal regions that are notexcluded, excluding the one or more determined focal regions of theplurality of focal regions that are excluded; and determining, based onthe focus values for the one or more of the plurality focal regions thatare not excluded and the second focus values, whether a scene in a fieldof capture of the camera has changed.
 2. The method of claim 1, whereindetermining whether to trigger the AF operation further comprises:determining a first overall focus value from the focus values for theone or more of the plurality of focal regions that are not excluded;determining a second overall focus value from the second focus values;and comparing the first overall focus value to the second overall focusvalue, wherein determining whether the scene has changed is based on thecomparison.
 3. The method of claim 1, further comprising: determining,for a focal region, a focal length at which a corresponding portion of ascene in a field of capture of the camera is in focus; and comparing thedetermined focal length to a reference focal length, wherein determiningwhether to exclude the focal region is based on the comparison.
 4. Themethod of claim 3, wherein a focus value for the focal length at whichthe corresponding portion of the scene is in focus is one from the groupconsisting of: a highest contrast of contrasts for a plurality of focallengths of the camera; and a phase difference (PD) closest to zero ofPDs for the plurality of focal lengths of the camera.
 5. The method ofclaim 1, further comprising generating a depth map of focal lengths forthe plurality of focal regions, wherein the determining of one or morefocal regions to exclude is based on focal lengths in the generateddepth map.
 6. The method of claim 1, further comprising: excluding theone or more determined focal regions from being used in performing an AFoperation; and performing the AF operation, comprising using the one ormore of the plurality of focal regions that are not excluded todetermine a new focal length for the camera.
 7. The method of claim 1,wherein determining whether to trigger the AF operation comprisesdetermining to trigger the AF operation based the one or more of thefocal regions that are not excluded.
 8. The method of claim 1, furthercomprising: triggering the AF operation when the scene in the field ofview of capture has changed.
 9. A device configured to perform anautofocus (AF) operation for a camera, comprising: one or moreprocessors; and a memory coupled to the one or more processors andcomprising instructions that, when executed by the one or moreprocessors, cause the device to: determine a focus value for each of aplurality of focal regions for a camera set to a focal length;determine, based on the focus values, one or more focal regions of theplurality of focal regions to exclude from use in triggering orperforming an AF operation; and determine whether to trigger the AFoperation, wherein, to determine whether to trigger the AF operation,the instructions, when executed by the one or more processors, furthercause the device to: determine a second focus value for each of one ormore of the plurality of focal regions that are not excluded, excludingthe one or more determined focal regions of the plurality of focalregions that are excluded; and determine, based on the focus values forthe one or more of the plurality of focal regions that are not excludedand the second focus values, whether a scene in a field of capture ofthe camera has changed.
 10. The device of claim 9, wherein theinstructions, when executed by the one or more processors, further causethe device to: trigger the AF operation when the scene in the field ofview of capture has changed.
 11. The device of claim 10, wherein theinstructions, when executed by the one or more processors, further causethe device to: determine a first overall focus value from the focusvalues for the one or more of the plurality of focal regions that arenot excluded; determine a second overall focus value from the secondfocus values; and compare the first overall focus value to the secondoverall focus value, wherein determining whether the scene has changedis based on the comparison.
 12. The device of claim 9, wherein theinstructions, when executed by the one or more processors, further causethe device to: determine, for a focal region, a focal length at which acorresponding portion of a scene in a field of capture of the camera isin focus; and compare the determined focal length to a reference focallength, wherein determining whether to exclude the focal region is basedon the comparison.
 13. The device of claim 12, wherein a focus value forthe focal length at which the corresponding portion of the scenecaptured by the camera is in focus is one from the group consisting of:a highest contrast of contrasts for a plurality of focal lengths of thecamera; and a phase difference (PD) closest to zero of PDs for theplurality of focal lengths of the camera.
 14. The device of claim 9,wherein the instructions, when executed by the one or more processors,further cause the device to: generate a depth map of focal lengths forthe plurality of focal regions, wherein the determining of one or morefocal regions to exclude is based on focal lengths in the generateddepth map.
 15. The device of claim 9, wherein the instructions, whenexecuted by the one or more processors, further cause the device to:exclude the one or more determined focal regions from being used inperforming an AF operation; and perform the AF operation, comprisingusing the one or more of the plurality of focal regions that are notexcluded to determine a new focal length for the camera.
 16. The deviceof claim 9, wherein to determine whether to trigger the AF operationcomprises to determine to trigger the AF operation based the one or moreof the focal regions that are not excluded.
 17. A non-transitorycomputer-readable medium storing one or more programs containinginstructions that, when executed by one or more processors of a device,cause the device to: determine a focus value for each of a plurality offocal regions for a camera set to a focal length; determine, based onthe focus values, one or more focal regions of the plurality of focalregions to exclude from use in triggering or performing an autofocus(AF) operation; and determine whether to trigger the AF operation,wherein, to determine whether to trigger the AF operation, theinstructions, when executed by the one or more processors, further causethe device to: determine a second focus value for each of one or more ofthe plurality of focal regions that are not excluded, excluding the oneor more determined focal regions of the plurality of focal regions thatare excluded; and determine, based on the focus values for the one ormore of the plurality of focal regions that are not excluded and thesecond focus values, whether a scene in a field of capture of the camerahas changed.
 18. The non-transitory computer-readable medium of claim17, wherein the instructions, when executed by the one or moreprocessors, further cause the device to: trigger the AF operation whenthe scene in the field of view of capture has changed.
 19. Thenon-transitory computer-readable medium of claim 18, wherein theinstructions, when executed by the one or more processors, further causethe device to: determine a first overall focus value from the focusvalues for the one or more of the plurality of focal regions that arenot excluded; determine a second overall focus value from the secondfocus values; and compare the first overall focus value to the secondoverall focus value, wherein determining whether the scene has changedis based on the comparison.
 20. The non-transitory computer-readablemedium of claim 17, wherein the instructions, when executed by the oneor more processors, further cause the device to: determine, for a focalregion, a focal length at which a corresponding portion of a scene in afield of capture of the camera is in focus; and compare the determinedfocal length to a reference focal length, wherein determining whether toexclude the focal region is based on the comparison.
 21. Thenon-transitory computer-readable medium of claim 17, wherein theinstructions, when executed by the one or more processors, further causethe device to: generate a depth map of focal lengths for the pluralityof focal regions, wherein the determining of one or more focal regionsto exclude is based on focal lengths in the generated depth map.
 22. Thenon-transitory computer-readable medium of claim 17, wherein theinstructions, when executed by the one or more processors, further causethe device to: exclude the one or more determined focal regions frombeing used in performing an AF operation; and perform the AF operation,comprising using the one or more of the plurality of focal regions thatare not excluded to determine a new focal length for the camera.
 23. Thenon-transitory computer-readable medium of claim 17, wherein todetermine whether to trigger the AF operation comprises to determine totrigger the AF operation based the one or more of the focal regions thatare not excluded.
 24. A device configured to perform an autofocus (AF)operation, comprising: means for determining a focus value for each of aplurality of focal regions for a camera set to a focal length; means fordetermining, based on the focus values, one or more focal regions of theplurality of focal regions to exclude from use in triggering orperforming an autofocus (AF) operation; and means for determiningwhether to trigger the AF operation, comprising: means for determining asecond focus value for each of one or more of the plurality of focalregions that are not excluded, excluding the one or more determinedfocal regions of the plurality of focal regions that are excluded; andmeans for determining, based on the focus values for the one or more ofthe plurality of focal regions that are not excluded and the secondfocus values, whether a scene in a field of capture of the camera haschanged.
 25. The device of claim 24, further comprising: means fortriggering the AF operation when the scene in the field of view ofcapture has changed.
 26. The device of claim 25, further comprising:means for determining a first overall focus value from the focus valuesfor the one or more of the plurality of focal regions that are notexcluded; means for determining a second overall focus value from thesecond focus values; and means for comparing the first overall focusvalue to the second overall focus value, wherein determining whether thescene has changed is based on the comparison.
 27. The device of claim24, further comprising: means for determining, for a focal region, afocal length at which a corresponding portion of a scene in a field ofcapture of the camera is in-focus; and means for comparing thedetermined focal length to a reference focal length, wherein determiningwhether to exclude the focal region is based on the comparison.
 28. Thedevice of claim 24, further comprising: means for generating a depth mapof focal lengths for the plurality of focal regions, wherein thedetermining of one or more focal regions to exclude is based on focallengths in the generated depth map.
 29. The device of claim 24, furthercomprising: means for excluding the one or more determined focal regionsfrom being used in performing an AF operation; and means for performingthe AF operation, comprising using the one or more of the plurality offocal regions that are not excluded to determine a new focal length forthe camera.
 30. The device of claim 24, wherein the means fordetermining whether to trigger the AF operation comprises means fordetermining to trigger the AF operation based the one or more of thefocal regions that are not excluded.